Efficient SPARQL Query Processing via Map-Reduce-Merge

The move towards a “semantic web” is driving the need for efficient querying ability over large datasets consisting of statements about web resources. RDF is a set of standards for describing and modeling data and is the backbone of the semantic web technologies. RDF datasets can be very large, and often are subject to complex queries with the intent of extracting and infering otherwise unseen connections within the data. MapReduce is a framework that allows for simplified development of programs for processing large data sets in a distrubuted, parallel, fault tolerant fashion. Map-Reduce provides many of the required features to support the type of querying needed in the semantic web, but historically has suffered from a lack of a natural way to process joins a critical component to RDF query processing. This paper presents a set of algorithms to support efficient processing of the core of SPARQL, an RDF query language, over an extension of Map-Reduce. A simple implementation of these algorithms is presented, and preliminary results are documented.